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The  Impact  of  Alternate  Specifications  of  Business  Performance: 
"Goal -Centered"  versus  "Goal -Independent"  Perspectives 

Summary 

This  paper  evaluates  the  impact  of  two  alternate  specifications  of 
business  performance  in  strategy  research,  namely  "goal-centered"  versus 
"goal-independent"  perspectives.  The  former  explicitly  recognizes  the  role 
of  organizational  goals  in  the  measurement  of  performance,  while  the 
latter  is  based  on  an  external,  universal  scheme.  The  impact  of  choosing 
one  over  the  other  is  evaluated  using  both  statistical  and  theoretical 
criteria.  Based  on  a  two-phased  research  design  involving  field  and 
simulated  data,  this  paper  finds  strong  support  for  the  equivalence  of  the 
two  perspectives.  Implications  and  directions  for  future  research  are 
developed. 


INTRODUCTION 

Organizational  performance  or  its  broader  manifestation, 
organizational  effectiveness,  is  a  topic  of  interest  to  researchers  in 
several  disciplines  (such  as  population  ecology,  macro-organization 
theory,  micro-organization  theory,  finance,  industrial  organization 
economics)  including  strategic  management.  Thus,  while  a  search  for  an 
universal  model  of  organizational  effectiveness  has  been  unsuccessful 
(Kanter  and  Brinkerhoff,  1981;  Cameron  and  Whetten,  1983a),  researchers 
Increasingly  favor  discipline-specific  approaches.  Cameron  and  Whetten 
note  that  "As  a  construct,  organizational  effectiveness  is  similar  to  an 
untapped  terrain,  where  the  responsibility  lies  with  the  researcher  to 
chart  it"  (1983a;  p.  20).  Hofer  echoes  a  similar  view:  "it  seems  clear 
that  different  fields  of  study  will  and  should  use  different  measures  of 
organizational  performance  because  of  the  differences  in  their  research 
questions,  as  well  as  their  ultimate  purposes"  (1983;  pp.  44-45). 
Consequently,  a  major  task,  for  strategy  researchers  is  to  conceptualize 
the  organizational  performance  construct  (and  develop  appropriate 
measurement  schemes)  consistent  with  the  fundamental  premise  of  the  field. 

From  the  perspective  of  strategic  management,  any  conceptualization 
of  performance,  should  at  a  minimum,  reflect  the  level  of  goal-attainment 
since  it  is  the  true  test  of  any  strategy.  Most  researchers  subscribe  to  a 
view  that  strategies  are  developed  (or,  formed)  to  achieve  organization- 
specific  goals,  namely  objectives  (Schendel  and  Hofer,  1979).  Thus,  there 
is  atleast  Impllct  acceptance  that  a  scheme  for  the  measurement  of 
performance,  which  reflects  the  effectiveness  of  strategies,  should  be 
anchored  1n  relation  to  the  organizational  goal -structure.  This  is  labeled 
here  as  a  "goal-centered"  perspective. 


But,  the  empirical  research  stream  has  largely  adopted  universal 
schemes  that  do  not  directly  reflect  organization-specific  goal  functions 
(Downey  and  Ireland,  1985),  that  is  termed  here  as  a  "goal-independent" 
perspective.  The  implicit  rationale  appears  to  be  that  the  use  of  a 
goal-independent  perspective  is  an  adequate  approximation  of  the  goal- 
centered  perspective,  in  spite  of  arguments  for  considering  organizational 
goal  structures  in  the  operational ization  of  performance  (see  for 
instance.  Steers,  1975;  Kirchoff,  1977;  Schendel  and  Hofer,  1979).   The 
purpose  of  this  paper  is  to  empirically  evaluate  the  specific  implications 
of  adopting  the  goal-independent  measurement  approach  when  the  theoretical 
stance  favors  the  goal-centered  approach.  In  other  words,  this  study 
formally  tests  the  equivalence  of  the  two  measurement  approaches  using 
both  statistical  and  theoretical  criteria. 

THEORETICAL  PERSPECTIVES 
Goal-centered  Versus  Goal -independent  Perspectives 

Researchers  have  long  debated  the  relative  merits  of  the  goal- 
centered  perspective  over  other  perspectives  including  the  goal- 
independent  perspective  (also  termed  as  "evaluative"  or  the  "natural 
systems"  approach)'  .  The  use  of  a  goal-centered  perspective  has  its 
origins  in  Etzioni's  (1964)  classical  work  on  organizations.  His  view  of 
effectiveness  as  the  degree  to  which  an  organization  realizes  its  goals, 
has  been  subsequently  refined  by  Price  (1972)  in  terms  of  derived  goals 
(i.e.,  judgements  by  an  individual  or  a  group  external  to  the  focal 
organization)  and  prescribed  goals  (i.e.,  those  that  the  focal 
organization  deflnps  for  itself).  While  there  are  compelling  reasons  for 
conceptualizing  performance  in  terms  of  a  goal-based  approach,  the  major 
operational  problems  are  to  be  recognized.  These  include:  (a)  the 


Identification  of  what  constitutes  the  particular  organization's  goals 
since  the  stated  goals  may  not  be  its  operative  goals;  (b)  the  inability 
of  such  a  system  to  compare  diverse  organizations  along  a  common  framework 
(Yuchtman  and  Seashore.  1967);  and  (c)  the  multiplicity  of  goals  with 
shifting  criteria  for  goal -tradeoff s  (Cyert  and  March,  1963). 

Thus,  an  alternative  of  developing  a  common  evaluative  framework 
(i.e.,  goal-Independent  perspective)  had  intrinsic  appeal,  and  many 
researchers  (e.g.,  Mahoney,  1967;  Mahoney  and  Weitzel,  1969;  Pickle  and 
Friedlander,  1967)  searched  for  common  measures  that  can  be  used  for 
evaluating  a  diverse  set  of  organizations.  This  approach  (summarized  by 
Campbell  and  others,  1974)  has  given  rise  to  several  measures  such  as: 
overall  effectiveness,  productivity,  efficiency,  profit  and  growth,  but  it 
fails  to  reflect  the  context-specific  importance  of  some  over  the  others. 

While  there  are  strong  points  favoring  each,  the  larger  question  of 
the  superiority  of  one  over  the  other  has  not  been  (and  may  be  never  will 
be)  resolved.  For  some  research  goals,  it  may  be  useful  to  adopt  an 
evaluative  perspective,  while  for  others,  it  may  be  both  appropriate  and 
critical  to  adopt  a  goal-centered  approach.  But,  it  appears  that  for  most 
strategy  research  questions  —  that  seek  to  evaluate  the  effectiveness  of 
strategies,  a  goal-centered  perspective  may  be  more  relevant.  The  debate 
over  the  appropriateness  of  one  perspective  over  the  other  is  discussed 
below,  with  specific  reference  to  strategic  management  research. 


The  Debate:  Its  Importance  for  Strategic  Management  Research 

Theoretical  Arguments.  Fundamental  to  strategic  management  is  the 
notion  of  strategic  choice  (Andrews,  1980;  Bourgeois,  1985;  Chandler, 
1962;  Child,  1972;  Thompson.  1967)  with  a  recognition  that  organizations 
have  different  and  multiple  goals  (although  they  may  not  be  explicit)  and 
that  strategies  are  developed  to  achieve  these  goals.  Although  there  may 
not  be  widespread  agreement  on  the  definitions  of  the  various  constructs, 
most  researchers  would  agree  that  the  more  important  constructs  are: 
goals,  strategies,  and  performance  (or,  more  generically  termed  as  ends, 
means,  and  outcomes).  For  instance,  Chandler's  classic  definition  of 
strategy  as  "...  the  determination  of  the  basic  long-range  goals  and 
objectives  of  an  enterprise,  and  the  adoption  of  courses  of  action  and  the 
allocation  of  resources  necessary  for  carrying  out  these  goals"  (1962; 
p. 16)  distinguishes  between  goals  (objectives,  ends)  from  strategies 
(actions,  means).  A  similar  perspective  characterizes  the  Harvard  Business 
School's  business  policy  paradigm  (see  for  instance,  Andrews,  1980). 

The  more  recent  strategic  management  research  paradigm  (Schendel  and 
Hofer,  1979;  p. 15)  is  explicit  in  separating  between  goal s  (formulation, 
structure)  from  strategies  (formulation,  evaluation,  and  implementation) 
adopted  to  achieve  these  goals,  i.e.,  performance.  Within  this  paradigm, 
performance  assessment  is  explicitly  linked  to  the  goal-structure  through 
the  "test  of  consistency"  (1979;  p. 15);  and  Rumelt  (1979)  discusses 
goal-consistency  test  as  a  specific  criterion  of  strategy  evaluation. 
Indeed,  there  exists  a  set  of  axiomatic  interl inkages  among  organizational 
goals,  strategies,  and  performance  (see  also,  MacCrimmon,  forthcoming). 


Thus,  there  are  strong  theoretical  reasons  to  anchor  the 
specification  of  performance  in  relation  to  the  specific  goal-structure  of 
the  focal  organization.  In  this  vein,  Steers  noted,  "it  would  appear  that 
attempts  to  measure  effectiveness  should  be  made  with  reference  to  the 
operative  goals  that  an  organization  is  pursuing;  ...Once  the  actual 
behavioral  Intentions  of  an  organization  are  identified,  it  is  then 
possible  to  ascertain  the  degree  to  which  those  intentions  are  being 
realized.  Such  an  approach  reduces  our  value  premises  about  what  an 
organization  should  be  doing  and  relies  instead  on  what  jjt  Is  actually 
trying  to  do"  (1975;  p.  555).  The  underlying  argument  1s  that  research 
efforts  at  operational izing  organizational  performance  should  recognize 
that  "[T]he  equal  treatment  accorded  the  various  criteria  ..  introduces  a 
..  source  of  error  in  measurement.  While  the  identification  of  such 
weights  places  an  added  burden  on  researchers"  efforts  at  parsimony,  such 
efforts  should  yield  far  more  valid  and  representative  data  than  most 
present  attempts"  (Steers,  1975;  p.  555). 

Other  researchers  have  echoed  similar  views.  For  instance.  Kirchoff 
(1977)  noted  that  "..  There  is  no  ultimate  criterion  of  effectiveness. 
Complex  organizations  pursue  multiple  goals.  Real  effectiveness  can  only 
be  measured  relative  to  a  particular  set  of  derived  or  prescribed  goals 
(1977;  p.  352).  He  went  on  to  conclude  that  "In  summary,  empirical 
research  on  policy  must  incorporate  more  complex  measures  of 
organizational  effectiveness...  models  of  effectiveness  expressing  various 
derived  goals  of  the  organization  represent  the  future  of  policy  research" 
(1977;  p.  354;  emphasis  added);  Wood  and  LaForge  at  the  end  of  their  study 
on  the  performance  Impact  of  strategic  planning  practices  noted  that  ".. 
it  probably  Is  time  for  researchers  in  this  area  to  abandon  the 
smorgasbord  use  of  financial  measures  as  dependent  variables  and  try  to 


match  up  the  appropriate  performance  criteria  with  the  primary  objectives 
of  the  organization  being  studied"  (1979;  p.  526);  and  King's  (1983) 
approach  to  the  evaluation  of  organizational  strategic  planning  systems 
and  strategic  choices  calls  for  the  recognition  of  the  unique  goals  of  the 
firm. 

Empirical  Reality.  In  spite  of  strong  theoretical  arguments,  the 
empirical  studies  have  adopted  a  goal-independent  perspective.  The  term, 
"empirical"  is  used  to  denote  those  that  follow  the  "variance  paradigm" 
(Mohr,  1982)  —  where  the  variance  in  the  dependent  variable,  say 
performance,  Is  explained  (and/or  predicted)  by  a  set  of  theoretically 
chosen  Independent  (say,  strategy)  variables.  In  this  tradition,  the 
typical  approach  has  been  to  employ  a  "generic  battery  of  performance 
Indicators',  often  chosen  by  the  researcher  under  an  implicit  assumption 
that  they  apply  to  "nearly  all  organizations"  In  the  research  sample. 
Hofer's  observations  are  particularly  telling:   "the  vast  majority  of  the 
research  studies  done  over  the  past  two  decades  have  relied  almost 
exclusively  on  performance  measures  that  have  been  "handed  down"  from 
industry  practice. ...  almost  all  of  them  were  developed  for  different 
purposes  than  for  doing  academic  research  on  the  relative  performance  of 
organizations  and  the  relative  effectiveness  and  efficiency  of  different 
organizational  strategies,  structures,  and  processes"  (1983;  p.  45-46). 

A  stronger  criticism  of  our  failure  to  link  performance  measurement 
to  the  goal  structure  is  made  by  Downey  and  Ireland  (1985).  Based  on  an 
analysis  of  6?  strategy  studies  published  over  a  five  year  period 
(1980-1985),  they  concluded  that  "in  twenty-nine  studies,  no  attempt  was 
made  to  explain  why  they  measured  organizational  outcomes  as  they  did;  In 
twenty-three  studies,  organizational  outcomes  were  measured  as  they  were 
because  someone  else  had  done  so  previously;  ....and  in  only  one  study  was 


8 

an  attempt  made  to  weight  the  researcher's  chosen  outcome  measures  by  the 
strategic  decision  makers"  purposes.."  (1985;  p.  9). 

Based  on  this  review,  they  admonished  strategy  researchers  for 
measuring  "outcomes"  without  tailoring  them  to  the  organizational 
intentions  or  goals.  They  specifically  called  upon  strategy  researchers  to 
"recognize  that  the  measurement  of  outcomes  in  policy  research  must  start 
with  the  policy  makers"  purposes..."  and  cautioned  that  "to  use  the  same 
set  of  outcome  measures,  no  matter  what  they  are,  to  assess  the  strategic 
outcomes  of  firms  Is  misleading"  (1985;  p.  14,  emphasis  added). 

Research  Question 

Two  general  questions  emerge  at  this  juncture:  One  1s:  what  explains 
the  glaring  divergence  between  the  theoretical  position  and  empirical 
reality?,  and  the  other  is:  why  is  this  important? 

The  first  question  can  be  answered  only  indirectly,  since  previous 
research  studies  provide  little  information  to  justify  their  choice 
(Downey  and  Ireland,  1985).   One  plausible  answer  is  that  researchers  have 
chosen  the  relatively  easier  approach  of  a  goal-independent  perspective 
with  an  impl icit  reasoning  that  the  research  results  are  unaffected  by  the 
choice  of  measurement  scheme.  But,  a  more  disturbing  observation  from 
Downey  and  Ireland's  analysis  is  the  general  lack  of  attention  provided  to 
the  discussion  and  justification  of  the  dependent  variable,  namely 
performance,  which  Is  supposedly  explained  (and/or  predicted)  by  the 
researchers"  theoretically-chosen  set  of  independent  variables  and 
corresponding  operational  measures.  A  practical  explanation  lies  in  the 
fact  that  a  greater  proportion  of  strategy  research  in  recent  years  is 
based  on  databases  such  as  the  PIMS  (Ramanujam  and  Venkatraman,  1984).  or 
COMPUSTAT  --  where  relevant  data  on  the  goal-structure  is  not  available. 


The  second  question  is  a  more  pertinent  one,  namely:  What  is  the 
impact  of  measuring  organizational  performance  independent  of  the 
organizational  goal  function  when  the  theoretical  treatment  is  clearly 
anchored  in  terms  of  organization-specific  goals?  Addressing  this  question 
is  in  the  spirit  of  construct  validation  that  is  central  to  theory- 
building  and  theory-testing  (Cronbach,  1971).  Specifically,  our  focus  is 
to  evaluate  whether  the  use  of  a  goal -independent  measurement  scheme  for 
the  performance  construct  conceived  along  a  goal-centered  approach  affect 
the  empirical  relationships? 

This  question  deserves  our  close  attention  because  the  extreme 
implication  of  Downey  and  Ireland's  argument  is  that  a  large  proportion  of 
research  findings  that  have  been  accumulated  over  the  last  decade  using 
the  goal-independent  measurement  of  perfoi^mance  could  be  invalidated  (or, 
rejected)  if  replicated  using  a  goal-centered  measurement  scheme.  Given 
our  collective  interest  in  developing  a  cumulative  theory-building 
perspective,  it  is  important  to  review  and  critically  evaluate  the 
foundations  supporting  our  research  findings. 

This  research  question  is  stated  as  a  null  hypothesis  (Hp )  pitted 

against  an  alternative  hypothesis  (H,  )  as  follows: 

Ho  :  Evaluations  of  organizational  performance  using 
the  two  approaches  --  goal-centered  and  goal-independent  — 
will  be  equivalent. 

H.  :  Evaluations  will  not  be  equivalent,  implying  that 
the  theoretical  position  of  a  goal-centered  perspective 
should  be  directly  reflected  in  the  measurement  scheme. 

This  hypolh^sli  is  tested  using  two  complementary  approaches:  (a) 
statistical  equivalence  —  reflecting  the  level  of  statistical  association 
between  the  two  measurements;  and  (b)  substantive  (i.e.,  theoretical) 

equivalence  --  reflecting  the  level  of  consistency  in  the  results  of  a  set 
of  empirical  relationships  using  the  two  measures. 
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METHODS  AND  RESULTS 
Overview 

This  study  is  based  on  a  set  of  analysis  conducted  across  two 
sequential  phases.  The  first  phase  focused  on  an  empirical  evaluation  of 
the  equivalence  of  the  two  measurement  approaches  based  on  field  data 
obtained  from  a  sample  of  business  units.  The  general izabi 1 ity  of  the 
results  from  this  phase  is  tested  in  the  second  phase  using  a  Monte  Carlo 
simulation  design  which  accommodates  the  possibility  of  experimentally 
varying  the  key  parameters  of  the  model  over  a  meaningful  range. 

Measurement  of  Business  Performance. 

A  comprehensive  measurement  scheme  of  business  performance^  is 
broader  than  the  choice  between  goal-centered  versus  goal-independent 
approaches.  Accordingly,  the  performance  construct  is  circumscribed  based 
on  the  guidelines  provided  by  Cameron  and  Whetten  (1983b)  as  shown  in 
Table  1.  Subsequently,  following  Steers"  (1977)  guideline  calling  for 
multiple  indicators  of  the  measure,  five  operational  indicators  were 
chosen.  This  choice  was  guided  by  Cyert  and  March's  argument  that  "In 
general,  we  have  observed  that  we  can  represent  organizational  goals 
reasonably  well  by  using  five  different  goals."  (1963;  p. 40).  The 
indicators  chosen  to  reflect  the  performance  construct  along  with 
corresponding  reference  to  the  Cyert  and  March  goal-structure  is  provided 
in  Table  2. 

(INSERT  TABLES  1  AND  2  ABOUT  HERE) 

Mathematical  Specifications.  The  two  different  perspectives  are 
sped  fled  as  fol  lows: 

Goal-independent  Perspective:  Pg i  =1  x^  (1) 

Goal-centered  Perspective:    P„  ^  =Iw,  .x, (2) 
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where, 

Pg  i  =    Performance  measure  under  the  goal-independent 
perspective  reflecting  a  simple  summated  scale; 

Pg c  =    Performance  measurement  under  the  goal-centered  perspective 
reflecting  a  weighted  summated  scale. 

Xi   =    individual  item  scores  on  a  five  point  Likert-type  scale 
ranging  from  5=highly  satisfied  to  l=highly  dissatisfied. 

Wi  =  the  weight  assigned  by  the  responding  executive  to  each 
performance  item  on  a  rank-ordered  scale  from  5=highest 
importance  and  l=lowest  importance. 

The  specification  of  a  goal-centered  perspective  is  consistent  with 
Steers  (1975),  who  noted  that:  "..  new  attempts  ,.  could  include  efforts 
to  account  for  differential  weights  on  the  various  evaluation  criteria  to 
reflect  different  valences  attached  to  each  goal."  (1975;  p. 555).  He  went 
on  to  argue  that  "Few  organizations  pursue  their  numerous  operative  goals 
with  equal  vigor  or  resources.  A  business  organization,  for  example,  may 
place  five  times  the  weight  on  the  pursuit  of  profit  as  it  does  on  the 
pursuit  of  community  welfare.."  (1975;  p. 555). 

But,  it  is  unclear  as  to  whether  the  weighting  scheme  should  be 
based  on  an  interval  scale  or  a  ranking  scale.  This  is  because  the  former 
reflects  a  "preference  independent"  perspective  (namely,  the  importance 
attached  to  one  goal  is  independent  of  the  importance  attached  to 
another),  while  the  latter  reflects  a  "preference  dependent"  perspective 
(namely,  the  importance  attached  to  one  constrains  the  importance  attached 
to  the  others  in  the  goal-set;  or  recognizes  that  trade-offs  are  necessary 
given  resource  constraints).  Following  Cyert  and  March  (1963),  and  Simon 
(1964),  the  rationale  is  that  the  weighting  scheme  should  reflect  a 
constrained,  preference  dependent  perspective.  Hence,  a  scale  that  ranks 
the  five  objectives  was  chosen. 
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Phase  1 

Data.  The  data  was  collected  in  two  stages  during  1984-85  using  a 
self-administered,  structured  questionnaire  as  a  part  of  a  larger  research 
project  that  required  two  stages  of  data  collection.  In  the  two  mailings, 
presidents/chief  executive  officers  of  700  strategic  business  units 
(divisions)  randomly  chosen  from  the  Directory  of  Corporate  Affiliations 
(250  in  the  first  phase  and  450  in  the  second  phase;  no  overlap)  were 
requested  to  participate  1n  the  study.  A  total  of  203  usable  responses 
were  obtained  representing  an  effective  response  rate  of  over  30%  (after 
consideration?  of  declines  because  of  company  policy).  No  significant 
stage-specific  sample-bias  could  be  found  as  the  two  samples  were  similar 
along  a  set  of  strategic  characteri stics-\  and  the  response  rate  is 
higher  than  the  typical  response  obtained  from  such  target  populations 
(Gaedke  and  Tootellan,  1976).  Relevant  descriptive  statistics  and  a  matrix 
of  zero-order  correlations  among  the  five  variables  (x^  s)  are  provided 
in  Table  3. 

(INSERT  TABLE  3  ABOUT  HERE) 

Analysis  for  Statistical  Equivalence.  The  simplest  and  the  most 
logical  approach  to  test  for  statistical  equivalence  between  P^ i  and 
Pg^  1s  to  calculate  the  level  of  association  between  them  using 
Pearson's  zero-order  correlation  coefficient  (r)  between  the  two 
measurements  defined  by  equations  (1)  and  (2).  The  value  of  (r)  was  0.80, 
which  is  statistically  different  from  zero  at  p<.01.  But,  this  is  not 
surprising  since  equations  (1)  and  (2)  are  arithmetically  related. 

Hence,  an  alternate  non-parametric  test  statistic  given  by  the 
Kendall's  copfflclent  of  concordance  (W)  between  the  two  measurement 
schemes  was  employed.  Since  this  Is  an  Indication  of  the  extent  to  which 
there  Is  concordance  between  the  ratings  (i.e.,  rankings)  of  two  different 
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judges  (or  measurement  schemes) ,  it  is  not  influenced  by  the  underlying 
arithmetic  relationships  between  them.  The  value  of  W  was  0.9615,  with  a 
corresponding  chi-squared  statistic  of  369.20  (df:198),  p<.001.  The 
implications  is  that  if  one  researcher  had  adopted  P,  j  and  the  other  had 
used  Pg ^  to  rank  a  set  of  firms  in  terms  of  their  performance,  their 
ranking  scheme  would  be  statistically  identical.  This  provides  strong 
support  for  the  statistical  equivalence  of  the  two  measurement  schemes. 

Substantive  (Theoretical)  Equivalence.  Testing  for  statistical 
equivalence  of  measurements  is  a  necessary  first  step,  but  is  not  a 
sufficient  test  for  establishing  equivalence.  This  is  because  it  provides 
support  for  convergent  validity  of  measurements  but  not  for  the  more 
important  criterion,  namely  theoretical  (or,  nomological  validity).  More 
specifically,  it  is  unclear  whether  a  set  of  empirical  relationships  (say, 
with  strategy  constructs)  obtained  using  one  measurement  scheme  would  be 
consistent  with  a  corresponding  set  of  results  obtained  using  the  other. 
This  requires  an  assessment  that  focuses  on  the  theoretical  equivalence, 
or  "nomological  validation"  of  measurements  (Campbell,  1960;  Cronbach  and 
Meehl,  1955). 

To  assess  their  nomological  validity,  the  measures  P^ i  and  P^ ^ 
were  each  correlated  with  measures  of  six  traits  of  a  construct  termed  as 
"strategic  orientation"  —  which  reflect  key  characteristics  of 
business-level  strategic  posture.  The  construct  definitions  of  the  six 
traits  are  provided  in  Table  4,  and  the  operational  measures  have  been 
shown  to  satisfy  Important  measurement  criteria  such  as  unidimensional i ty, 
reliability,  convergent  and  discriminant  validity  ( Venkatraman ,  1985). 
Before  we  Interprpt  the  values  of  the  correlations,  it  is  important  to 
recognize  that  the  magnitude  and  significance  of  each  of  the  correlations 
between  strategy  traits  and  performance  is  only  partly  meaningful  without 
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the  broader  environmental  context.  Thus,  the  focus  is  not  on  the  value  of 
the  correlation  coefficients  per  se,  but  on  the  consistency  of  results 
across  the  two  performance  measurement  schemes.  The  correlation 
coefficients  between  each  measurement  scheme  and  the  six  dimensions  of 
strategic  orientation  are  summarized  in  Table  4,  which  also  includes  the 
t-statistic  that  tests  for  the  statistical  significance  of  the  difference 
in  the  magnitude  of  the  two  correlation  coefficients. 

(INSERT  TABLE  4  ABOUT  HERE) 

Results.  Three  patterns  are  important  from  the  results  summarized 
in  Table  4.  First,  it  1s  evident  that  the  directional ity  of  the 
relationships  is  unchanged  across  the  two  measures.  In  four  cases,  the 
relationship  between  the  strategy  dimension  and  the  two  performance 
measures  is  consistently  positive,  and  in  the  two  other  cases,  it  is 
consistently  negative.  Second,  the  statistical  support  for  the 
correlational-type  hypotheses  is  consistent  in  five  of  the  six  cases.  More 
specifically,  except  for  minor  shifts  in  the  level  of  statistical 
significance,  the  final  conclusion  would  have  been  the  same  in  all  cases 
except  the  relationship  between  the  proacti veness  dimension  of  strategy 
and  performance.  In  this  case,  had  one  hypothesized  a  positive  and 
significant  association,  this  would  have  been  marginally  accepted  (p<.10) 
under  the  goal-independent  perspective,  while  it  would  have  been  rejected 
under  the  goal-centered  perspective.  Third,  the  magnitude  of  associational 
effects  between  strategy  and  performance  is  not  statistically  different 
across  the  two  measurement  schemes  in  five  of  six  cases  as  indicated  by 
the  Insignificant  t-statistic.   In  the  remaining  case,  1t  is  marginally 
significant  (p^, 10). 

Discussion  and  Unresolved  Issues.  Thus,  one  can  generally  conclude 
that  the  empirical  results  supporting  the  underlying  theoretical 
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relationships  between  strategy  and  performance  would  have  been  generally 
the  same  irrespective  of  the  choice  of  performance  measurement  scheme. 
Consequently,  we  cannot  reject  the  null  hypothesis  of  the  equivalence  of 
the  measures.  It  appears  that  the  simpler,  unweighted  scheme  of 
performance  measurement,  Pg ^  is  statistically  and  empirically  equivalent 
to  the  more  complex  weighted  scheme,  P, ^  . 

However,  two  issues  are  unresolved  at  this  phase  of  research.  They 
are:  (a)  the  general izabi 1 Ity  of  the  research  results;  and  (b)  sensitivity 
of  the  results  to  the  specification  of  goal-centered  performance  as  in 
equation  (2).  The  first  Issue  reflects  the  underlying  question  —  what  can 
be  said  about  the  equivalence  of  the  two  perspectives  beyond  this  study? 
Will  this  result  hold  for  a  different  sample  of  organizations  with  a 
different  pattern  of  goal-preference,  or  is  it  limited  only  to  the  sample 
of  business  unit  studied?  Although  every  effort  was  taken  in  the  data 
collection  plan  to  obtain  a  random  sample  of  business  units  from  the 
population,  there  Is  a  finite  chance  that  the  sample  may  not  be  truly 
representative  of  the  larger  population.  The  external  validity  concerns 
are  typically  approached  through  replications,  although  in  select  cases  it 
can  be  assessed  using  simulations,  as  done  here. 

The  second  issue  1s  with  the  operationalization  of  the  goal-centered 
perspective  using  a  mathematical  form  as  specified  in  equation  (2).  One 
can  reasonably  argue  that  the  operationalization  of  a  goal-centered 
perspective  1s  by  no  means  limited  to  the  specification  of  equation  (2), 
although  1t  may  be  commonly-  used.  For  Instance,  one  could  well  specify 
the  goal-centered  perspective  using  a  scale  which  reflects  the  magnitude 
of  differences  In  the  weights  attached  to  the  different  goals  rather  than 
weighting  by  the  rank-ordered  scale.   Both  these  concerns  are  addressed 
through  a  Monte  Carlo  simulatin  design  discussed  next. 
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Phase  2 

The  rationale  for  this  phase  is  derived  from  the  need  to  test  the 
stability  of  the  results  obtained  in  the  first  phase  for  different 
samples,  as  well  as  an  alternate  specification  of  the  goal-centered 
perspective.  Within  a  Monte  Carlo  design,  this  is  achieved  by 
systematically  varying  the  key  conditions,  namely,  patterns  of 
correlations  among  the  five  indicators  across  several  random  combinations 
of  weighting  schemes.  Thus,  it  serves  as  a  more  efficient  way  to  assess 
the  general izabi 1 Ity  of  the  results  than  several  replications  using 
f1eld-data.  The  use  of  simulation-based  design  is  appropriate  for 
assessing  the  general izabil ity  of  the  results  in  this  study  because  the 
focus  is  on  the  consistency  of  results  rather  than  on  the  examination  of 
theoretical  relationships.  The  two  issues  underlying  this  phase  are 
analyzed  and  discussed  separately.  The  general izabi 1 ity  of  the  results  to 
varying  characteristics  of  the  sample  is  addressed  first.  A  brief, 
non-technical  description  of  the  steps  involved  in  performing  the 
simulation  runs  is  provided  in  the  Appendix. 

Results.  Table  5  summarizes  the  central  results  of  the 
simulations'*.  Two  values  (Type  A  and  B)  are  of  importance  in  this  set 
of  analyses.   Type  A  Indicates  the  probability  of  committing  an  error  of 
accepting  In  an  incorrect  hypothesis,  namely  the  probability  of  accepting 
a  correlation  between  strategy  and  P^ ^  as  statistically  significant  at 
p<.05,  when  the  correlation  between  strategy  and  Pg ^  is  not 
statistically  significant  at  the  same  level.   It  is  important  to  note  that 
this  value  was  constant  at  2%  over  the  entire  range  of  the  simulation. 
The  Implication  Is  that  in  98%  of  cases,  the  results  of  testing  the 
relationship  between  strategy  and  performance  would  be  consistent  across 
the  two  measures. 
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The  value  of  Type  B  provides  a  complementary  result.  It  indicates 
the  percentage  of  replications  within  a  particular  setting,  where  the 
magnitude  of  the  correlations  between  (a)  P^  ,  and  S  and  (b)  P^ ^  and  S 
is  not  statistically  different.   The  value  of  Type  B  ranges  from  86  to  92 
in  the  six  runs  reported  in  Table  5,  and  ranged  from  82  to  96  with  a  mean 
value  of  90%,  over  the  entire  range  of  simultation  runs  as  described  in 
the  Appendix  (Step  10). 

(INSERT  TABLE  5  ABOUT  HERE) 

Now,  we  proceed  to  test  the  sensitivity  of  the  results  to  the 
specification  of  equation  (2).   For  this  purpose,  the  equation  (2)  was 
modified  with  an  additional  condition,  namely  that  ^Wi=100.  Thus,  the 
weighting  scheme  requires  the  allocation  of  100  points  to  a  set  of  five 
goals  (in  units  of  10  to  limit  the  number  of  possible  combinations  of 
allocations).  Six  different  allocation  schemes  were  considered  for  this 
set  of  analyses.  The  design  of  the  simulation  experiments  follows  the 
same  steps  as  outlined  in  the  appendix.  Table  6  provides  a  summary  of  this 
set  of  simulation  runs  for  the  six  values  of  the  entries  in  the 
correlation  matrix  as  before.  It  appears  that  on  average,  the  equivalence 
criterion  is  satisfied  in  90%  of  cases,  and  cosistent  with  the  results  in 
Table  5. 

(INSERT  TABLE  6  ABOUT  HERE) 

Discussion  The  values  of  Types  A  and  B  taken  together  indicate 
that  the  impact  of  not  adopting  a  goal-centered  perspective  is  not  as 
serious  as  one  would  have  been  led  to  believe.   Thus,  the  results  from 
this  phase  provide  additional  support  for  the  general i zabi 1 1 ty  of  the 
results  obtained  In  the  first  phase.  Again,  the  null  hypothesis  of 
equivalence  In  the  measurement  schemes  cannot  be  rejected.   Further,  the 
results  are  Insensitive  to  the  specification  of  equation  (2),  as  a  set  of 
analysis  using  a  magnitude  scale  provided  consistent  results. 
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IMPLICATIONS  AND  EXTENSIONS 

The  recent  decade  of  strategic  management  research  has  been  marked 
by  Increasing  attention  to  the  measurement  of  performance  (see  for 
Instance,  Chakravarthy,  1986;  Dess  and  Robinson,  1984;  Hofer,  1983;  Lenz, 
1981;  Lubatkin  and  Shrleves,  1986;  Venkatraman  and  Ramanujam,  1986;  1987; 
Woo  and  Wlllard.  1983).  However,  Downey  and  Ireland  (1985)  attacked  the 
fundamental  conceptual  foundation  of  performance  measurement  from  a 
strategic  management  perspective  by  calling  attention  to  the  fact  that 
performance  measurement  is  not  anchored  In  relation  to  the  organizational 
goal-structure.  This  paper  specifically  sought  to  assess  the  seriousness 
of  their  admonishment. 

Based  on  the  results  of  this  study,  we  provide  strong  support  that  a 
goal-independent  perspective  operational ized  as  a  simple,  unweighted 
scheme  is  both  statistically  and  empirically  equivalent  to  the  more 
complex  goal-cenetered  perspective  that  requires  different  types  of 
weighted  schemes.  In  the  first  phase  using  data  from  a  sample  of  business 
units,  we  found  that  except  for  minor  shifts  in  the  level  of  significance, 
the  two  schemes  were  equivalent.  This  result  was  tested  for 
general izabi 1 ity  in  the  second  phase  using  Monte  Carlo  simulations.  These 
results  provide  further  corroboration  of  equivalence.  It  was  found  that  on 
an  average,  the  results  of  a  test  between  strategy  and  performance  would 
be  the  same  across  the  two  schemes  in  atleast  90%  of  cases,  which  is 
considered  acceptable  given  the  random  model. 

The  specific  Implication  is  that  if  the  researcher  is  reasonably 
confident  of  the  adequacy  of  the  indicators  reflecting  the  performance 
construct,  then  additional  data  on  the  differential  weights  may  not  be 
necessary  for  most  cases.  Thus,  our  results  downplay  the  importance  of 
Steers'  call  for  obtaining  differential  weights  (1975,  p. 555)  as  well  as 
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mitigate  the  seriousness  of  Downey  and  Ireland's  criticism  that  the 
results  of  extant  strategy  research  may  be  misleading  since  it  has  been 
largely  tested  using  a  goal-independent  perspective. 

However,  a  word  of  caution  is  necessary  here.  The  results  do  not 
negate  a  fundamental  axiom  that  the  conceptual  definition  should  guide  the 
development  of  a  measurement  scheme.  This  is  because  the  complexities  and 
nuances  of  the  construct  definition  and  operationalization  (such  as  the 
dimensionality  of  the  construct  and  the  type  of  data)  cannot  be  addressed 
within  a  single  study.  Consequently,  the  one  general  (but  important) 
guideline  from  this  study  is  a  call  for  greater  discussion  of  the 
underlying  issues  of  conceptualization  and  measurement  of  performance 
construct  in  strategy  research.   Further,  it  is  important  to  reiterate 
that  no  one  operationalization  is  intrinsically  superior.  For  some 
research  purposes,  it  may  be  necessary  to  anchor  performance  measurement 
in  terms  of  an  external  (evaluative)  criteria  (such  as  the  stock-market 
indices),  while  for  others,  a  goal-centered  perspective  may  be  the  choice 
dictated  by  the  underlying  theory  to  be  tested.  So,  we  strongly  urge  that 
researchers  move  away  from  implicit  selection  towards  formal,  and  explicit 
discussion  and  justification  of  their  scheme  for  performance  measurement. 
This  would  certainly  complement  the  recent  increase  in  the  level  of 
attention  to  the  measurement  of  strategy  constructs  (see  Venkatraman  and 
Grant,  1986  for  a  review). 

Extensions 

This  research  study  can  be  viewed  in  a  context  broader  than  an 
assessment  of  measurement  equivalence.   Two  important  extensions  that  have 
important  theoretical  underpinnings  are  identified  and  discussed  below. 
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Manager-art 1 cu 1 ated  versus  Researcher-specified  Goals.  In  this 
research  study  (especially  the  first  phase),  we  pre-specified  a  list  of 
performance  indicators  and  requested  managers  to  assign  a  set  of  ranks 
reflecting  their  organizational  preference.  Thus,  the  domain  of  the 
goal-structure  of  the  responding  organization  was  bounded  by  the 
researcher's  prior  conception.  The  equivalence  of  the  two  perspectives 
were  evaluated  within  this  constraint.  A  radically  different  approach  to 
the  assessment  of  equivalence  calls  for  a  design  that  lets  the  managers  in 
a  subsample  to  Indicate  their  top  five  goals  as  well  as  their  relative 
weights.  This  data  can  be  used  to  construct  a  measure  of  goal-centered 
performance.  Simultaneously,  one  could  have  a  "control"  subsample,  that 
collects  the  data  based  on  researcher-specified  indices  to  obtain  an 
alternative  measure  of  goal-centered  performance.  By  comparing  the  results 
of  relationships  between  strategy  and  performance  (or,  even  incorporating 
environmental  context)  across  these  two,  (i.e.,  experimental  and  control 
samples),  important  insights  into  the  sensitivity  of  the  results  to  the 
distinction  between  manager-articulated  goals  and  researcher-specified 
goals  acn  be  systematically  ascertained.   This  would  go  a  long-way  in 
refining  the  theoretical  domain  of  the  performance  construct. 

Link  to  the  "Consensus"  Research  Stream.  Another  line  of  extension 
pertains  to  the  incorporation  of  weighting  schemes  of  multiple  managers. 
This  study,  by  virtue  of  its  exploratory  nature,  obtained  data  from  one 
senior-level  manager,  acting  as  an  "informant"  for  the  organizational 
unit.  Although  the  specific  instructions  called  for  the  executive  to 
assume  the  role  of  an  organizational  spokesperson  (or  informant),  the  need 
to  obtain  response?  from  multiple  informants  cannot  be  underemphasized. 
This  is  because  these  weights  reflect  managerial  judgements  and 
perceptions  rather  than  some  "objective"  preferences. 
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Over  the  last  few  years,  several  studies  have  focused  on  the 
relationship  between  "consensus"  in  strategy  and  goal  structure  on  the  one 
hand,  and  performance  on  the  other  (e.g.,  Bourgeois,  1980;  Dess,  1987; 
Hrebiniak  and  Snow,  1982).  These  studies  have  focused  on  the  degree  of 
consensus  in  the  independent  variables,  such  as  goals  or  strategies,  but 
not  in  the  dependent  variable,  namely  performance.  Indeed,  this  stream  is 
marked  by  its  use  of  a  goal-independent  operational izatlon  of 
performance.  It  would  be  useful  and  insightful  to  develop  a  measure  of 
goal-centered  performance  using  weights  from  multiple  managers  and  examine 
its  relationship  with  the  degree  of  "divergence-consensus"  in  their  goal 
structures.  Simultaneously,  the  equivalence  of  these  two  measurement 
perspectives  can  be  studied  using  the  weighting  schemes  provided  by 
multiple  key  managers. 

CONCLUSIONS 

An  important  research  issue  in  strategic  management  pertains  to  the 
conceptualization  and  measurement  of  business  performance  that  reflects 
the  specific  concerns  of  the  field.  Within  this  theme,  a  major  dichotomy 
relates  to  the  distinction  between  goal-centered  and  goal-independent 
perspectives.  While  the  theoretical  arguments  favor  the  former,  the 
empirical  stream  has  followed  the  latter.  This  research  study  evaluated 
their  equivalence.   Results  based  on  a  two-phased  study  provide  strong 
support  for  their  equivalence  implying  that  an  unweighted,  summated  scale 
is  an  adequate  approximation  of  the  more  complex,  weighted  scale.   Inspite 
of  such  strong  results,  it  is  necessary  that  researchers  explicitly 
circumscribe  thpir  operational i zation  scheme  in  their  theoretical 
perspective  rather  than  implicitly  choose  a  conveniently  available  scheme. 
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NOTES 

1.  For  a  detailed  discussion  on  alternative  approaches  to 
performance  assessment,  see  Campbell  and  others  (1977);  and  Ford  and 
Schellenberg  (1982). 

2.  The  term  business  performance  is  used  to  explicitly  highlight 
that  the  discussion  is  at  the  level  of  a  business  (unit)  of  the 
organization. 

3.  Strategic  characteristics  were  measured  in  terms  of  the 
importance  attached  to  key  business  goals.  The  two  samples  exhibited 
statistically  significant  profiles.  A  detailed  discussion  of  this  result, 
along  with  a  profile  of  the  sample  is  availabe  on  request. 

4.  Detailed  results  are  available  on  request. 
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Table  1:  Defining  the  Business  Performance  Construcf^ 


Guldel ine 


Circumscription 


1.  From  whose  perspective  is 
performance  being  judged? 


2.  On  what  domain  of  activity 
is  the  judgement  focused? 


3.  What  level  of  analysis  is 
used? 


4.  What  1s  the  purpose  of  the 
assessment? 


5.  What  time  frame  Is  employed? 


6.  What  type  of  data  are 
sought? 


7.  What  is  the  reference 
against  which  performance 
is  judged? 


The  key  informant  who  Is  a  member  of  the 
dominant  coalition  of  the  business  unit 
being  judged. 

Key  characteristics  of  business 
performance  that  can  be  used  for 
comparison  across  business  units  were 
considered. 

The  organizational  level  (as  opposed  to 
the  Individual  level)  of  analysis  was 
the  focus.  Within  the  organization,  the 
business  unit  level  of  the  hierarchy  was 
chosen  consistent  with  the  goals  of  the 
research  project. 

This  assessment  was  made  to  compare 
across  businesses  along  multiple 
dimensions  of  performance. 

Performance  criteria  reflected  static, 
current  levels.   The  focus  was  on 
ascertaining  the  degree  to  which 
different  business  units  differ  along 
the  key  criteria. 

Perceptual  data  on  the  different 
criteria  of  performance  were  sought 
using  questionnaire. 

Referent  1:  A  goal-independent 
referent,  namely  that  a  business  unit  is 
considered  to  be  the  best  performer  if 
it  scored  higher  (on  a  simple 
performance  scale)  than  other  units  In 
the  sample. 

Referent  2:  A  goal-centered  judgement, 
namely  that  a  business  unit  is 
considered  to  be  the  best  performer  if 
It  scored  higher  (on  a  weighted 
performance  scale,  I.e.,  performance 
score  X  rank  of  the  criterion)  than 
other  units  In  the  sample. 


PBased  on  Cameron  and  Whetten  (1983b). 


Indicator 

1. 

Sales 

2. 

Market  Share 

3. 

Cash  flow 

4. 

Return  on  Sales 

5. 

Return  on  Investment 
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Table  2:  Indicators  and  Their  Theoretical  Anchors 

Theoretical  Dimension'^ 

Sales  goal 

Market  share  goal 

Production  and  Inventory  goal 

Profit  goal 

Profit  Goal 


(Each  indicator  scaled  from  5  =  highly  satisfied  to  1  =  highly 
dissatisfied  and  ranked  from  5  =  most  important  to  the  business  unit  to 
1  =  least  important  to  the  business  unit.) 


^Based  on  Cyert  and  March's  (1963)  concept  of  organizational  goals. 


Table  3:  Descriptive  Statistics  and  Zero-order  Correlations 


Importance 
Performance    Weights   Satisf ication Zero-Order  Correlations^ 


No 

Indicator 

Mode 

Mean 

Sd 

1 

2 

3 

4     5 

1 

Sales 

1 

3.35 

1.05 

1.0 

2 

Market  Share 

2 

3.3 

0.97 

0.50 

1.0 

3 

Cash  Flow 

2 

3.60 

1.08 

0.31 

0.33 

1.0 

4 

Return  on 
Sales 

4 

3.14 

1.12 

0.38 

0.32 

0.60 

1.0 

5 

Return  on 
Investment 

5 

3.31 

1.25 

0.48 

0.34 

0.62 

.76  1.0 

PAll  correlation?  aro  significant  at  levels  better  than  p  <    .05. 
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Table  4:  Impact  of  Alternate  Performance  Measurements 


Correlation  Coefficients      t-Statistic^ 
With  for 

Strategy  Dimension  (and       (1)        (2)        Testing  Difference 
a  Working  Definition)        P,  ^        P, .,     Between  (1)  and  (2) 


1.  Aggressiveness:         (-)0.0987   (-)0.1003*        0.03 
aggressive  allocation 

of  resources  for 
seeking  market  share 

2.  Analysis:  0.1034*     0.0938*        0.21 
overall  problem-solving 

and  decision  making 
posture 

3.  Defensiveness:  0.1909***   0.0886*        1.82^ 
posture  toward  defending 

one' s  care  strengths 

4.  Futurity:  0.0365      0,0822         0.74 
degree  of  futurity 

reflected  in  current 
strategic  decisions 

5.  Proactiveness:  0.1034*     0.0688        0.59 
posture  towards 

participation  in 
emerging  markets 

6.  Riskiness:  (-)0.1511**  (-)0.1193**       0.52 
degree  of  risk 

taking  behavior  in 

organizational 

decision-making 


*p  <  . 

10 

**p  < 

.05 

***p  < 

:    .0 

P  Calculations  are  on  the  basis  that  the  two  correlations  are 
dependent  on  the  value  of  correlation  between  P^ ,  and  P, ^  .   See 
Bruning  and  K<nt7  (1968). 
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Table  5:  Results  of  Monte  Carlo  SlWIUlations 


1.  Representation  of  the  Input  Data  Matrix  for  Each  Run 


r  X 


1 


3 


x> 


Values  of  X^  based  on 
the  value  of  palrwise 
correlations  between 
Xi  Is  used  as  Input 


^   w. 


w 


3 


W.   W, 


Rank-ordered  weights 
randomly  generated. 


1 


Random  values 
for  strategy 
dimension 


100x11 


2.  Salient  Results' 

Run        R, 


1 
2 
3 

4 
5 
6 


Type  A 


0.1 

2 

0.2 

2 

0.3 

2 

0.4 

2 

0.5 

2 

0.6 

2 

Type  B 

92 
89 
88 
86 
88 
91 


Type  A  =  Probability  that  a  relationship  between  strategy  and  Pg ^  is 
significant  at  p  <  .05,  while  it  is  not  significant  using  Pg ^ 

Type  B  =  Probability  that  the  magnitude  of  the  two  relationships  i.e., 
between  (a)  S  and  P^  i  and  (b)  S  and  P,,  ^  are  not 
statistically  different. 


1.  The  entire  simulation  exercise  was  more  comprehensive  than  the  six  runs 
reported  here  as  outlined  In  the  Appendix.  Since  the  results  were,  in 
general,  stable  only  the  results  of  major  intermediate  steps  in  the 
analyses  are  reported.   In  the  overall  analysis,  the  value  for  Type  A  was 
constant  at  2%,  while  the  results  for  Type  B  fluctuated  between  82  and  96, 
with  a  mean  value  of  90%.   (Detailed  results  on  request) 
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•  Appendix:  Summary  Steps  of  Simulation  Runs 

The  key  steps  involved  in  the  Monte  Carlo  simulation  runs  is 
provided  below. 

Step  1.  A  matrix  of  pairwise  correlations  among  the  five  indicators 
of  the  performance  construct  was  specified  as  follows: 


R  = 


1.0 

I 

1.0 

J 

K 

1.0 

L 

M 

N 

1.0 

0 

P 

Q 

R 

1.0 


All  the  values  were  set  initially  at  0.10. 

Step  2.  Based  on  this  pattern  of  correlations,  a  random  sample  of 
size  n,=100  was  generated  using  subroutines  of  the  Numerical  Algorithm 
Group  program  library  as  implemented  on  a  Prime  mini  computer. 

Step  3.  For  each  case  in  the  sample,  a  random  vector  of  five  values 
that  reflect  the  rank-ordered  weights  as  well  as  a  random  number  for  the 
strategy  dimension  (with  mean  0  and  standard  deviation  unity)  were 
generated.  This  provides  one  input  data  matrix  of  size  100X11. 

Step  4.  The  two  performance  measures,  P, ^  and  P,  ^  were  computed 
in  line  with  equations  (1)  and  (2). 

Step  5.  Pearson's  zero-order  correlations  were  calculated  between 
(a)  Strategy  and  Pg  ^  and  (b)  Strategy  and  P.,  ^  ,  with  their  accompanying 
tests  for  statistical  significance  at  p<.05. 

Step  6.  If  (a)  was  statistically  significant,  but  (b)  was  not,  then 
it  was  termed  as  Type  (A). 

Step  7.  The  difference  in  the  magnitude  of  the  correlations  was 
tested  using  a  t-statistic.  If  the  magnitude  was  not  statistically 
different,  then  it  was  termed  as  Type  (B). 

Step  8.  Steps  2  through  7  were  repeated  100  times  to  provide  100 
replications  for  each  setting  of  the  correlation  matrix  specified  in  step 
1.  Thus,  n,=100. 

Step  9.  For  each  setting,  the  values  of  Type  (A)  and  Type  (B)  were 
computed  after  the  100  replications.  The  value  of  Type  (A)  is  expected  to 
be  close  to  zero,  while  the  value  of  Type  (B)  is  expected  to  be  close  to 
100  to  provide  the  strongest  possible  support  for  the  equivalence  of 
measurements . 


Step  10.  The  value  of  entries  in  the  matrix  specified  in  step  1  was 
increased  in  steps  of  0.1  for  each  entry,  keeping  the  others  constant,  and 
steps  2  through  9  were  repeated. 

0  i  , 
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